Audio Analysis of Human/Virtual-Human Interaction

The audio of the spoken dialogue between a human and a virtual human (VH) is analyzed to explore the impact of H-VH interaction. The goal is to determine if conversing with a VH can elicit detectable and systematic vocal changes. To study this topic, we examined the H-VH scenario of pharmacy students speaking with immersive VHs playing the role of patients. The audio analysis focused on the students’ reaction to scripted empathetic challenges. Empathetic challenges are VH-initiated dialogue designed to generate an unrehearsed affective response from the human. The responses were analyzed with software developed to analyze vocal patterns during H-VH conversations. The analysis shows that although some of the vocal changes are undetectable by the human ear, they do occur, are detectable by digital signal processing, and are consistent across participants groups. Further, these changes are correlated with known H-H conversation patterns.